Scenarios Analyses

1. Overview of scenario data

First, I look at how participants rated the appropriateness of each violation in each scenario, and the appropriateness of the different types of punishments in that scenario.

Data is displayed separately by treatment, i.e. whether the violation is done by a friend/non-friend, and weather the punishment is performed by a friend to a friend/non-friend. Treatments are between subjects.

Insights:
All violations are considered fairly inappropriate, with insulting a family member and stealing being the worst.
There is quite some variation in appropriateness of the different types of punishment.
There are no striking differences between treatments.

Stealing

Phone

Interrupt

Insult

Avoid

1.1. Correlation of appropriantess of violations and appropriateness of punishment.

Correlation between violation ratings and punishment ratings across scenarios for each punishment type. The expectation is: negative relationship for angry remark, avoidance and gossip, and positive for doing nothing . (cf Fig. 1 of Eriksonn 2021)

Rho are Spearman ranked correlation coefficients (calculated across treatments). For a full overview see the table (third panel)

Without separating by domain

Separating by domain

full table of correlation coefficients

punishment_type	domain	treatment	spearman_rho	n
agry_remark	insult	nf	-0.22	244
agry_remark	insult	f	-0.16	239
agry_remark	interrupt	nf	-0.08	241
agry_remark	interrupt	f	-0.18	245
agry_remark	online	nf	-0.23	240
agry_remark	online	f	-0.10	237
agry_remark	stealing	nf	-0.10	248
agry_remark	stealing	f	-0.10	248
agry_remark	texting	nf	-0.17	245
agry_remark	texting	f	-0.02	244
avoid	insult	nf	-0.10	243
avoid	insult	f	-0.05	239
avoid	interrupt	nf	0.12	241
avoid	interrupt	f	-0.07	245
avoid	online	nf	-0.14	240
avoid	online	f	0.23	238
avoid	stealing	nf	-0.13	248
avoid	stealing	f	-0.08	248
avoid	texting	nf	-0.13	245
avoid	texting	f	-0.07	244
do_nothing	insult	nf	0.35	244
do_nothing	insult	f	0.23	239
do_nothing	interrupt	nf	0.12	242
do_nothing	interrupt	f	0.16	245
do_nothing	online	nf	0.31	240
do_nothing	online	f	0.15	237
do_nothing	stealing	nf	0.33	248
do_nothing	stealing	f	0.31	248
do_nothing	texting	nf	0.21	245
do_nothing	texting	f	0.39	244
gossip	insult	nf	0.02	244
gossip	insult	f	-0.03	239
gossip	interrupt	nf	0.13	242
gossip	interrupt	f	0.07	244
gossip	online	nf	0.08	240
gossip	online	f	0.14	238
gossip	stealing	nf	0.11	248
gossip	stealing	f	0.08	248
gossip	texting	nf	-0.02	245
gossip	texting	f	0.09	244

2. Analysis of appropriateness of violations across all domains

Now I merge all the scenarios together.

Insight:

Overall appropriateness is quite low, with again no striking difference between treatments.

2.1 ANOVA

First I run an ANOVA to get an idea of differences across treatments and domains. The ANOVA showed that both treatment and domain significantly affected ratings, with a significant interaction indicating that the effect of treatment varied across domains.

##                    Df Sum Sq Mean Sq F value   Pr(>F)    
## treatment           1     12   12.50   9.080  0.00261 ** 
## domain              4    324   80.97  58.817  < 2e-16 ***
## treatment:domain    4     47   11.71   8.507 8.16e-07 ***
## Residuals        2433   3349    1.38                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

2.2 Regressions

I now run two multilevel linear regressions with participants as random intercept to zoom in on size and direction of efffects.

First, a linear regression predicting rating ~ treatment shows a small effect of treatment: violations by friends are considered slightly more appropriate.

If we include a test of interactions between treatment and domain, we can see that the effect is driven by the “exclude from online chat scenario:

“One student who is part of your friends/others group (Student A) excluded another student, who is also a part of your friends group, from a group chat on WhatsApp.”

Rating ~ Treatment

	rating
Predictors	Estimates	CI	p
(Intercept)	1.08	1.00 – 1.16	<0.001
treatmentf	0.14	0.05 – 0.23	0.003
Random Effects
σ²	1.26
τ₀₀ _ID2	0.27
ICC	0.18
N _ID2	496
Observations	2443
Marginal R² / Conditional R²	0.003 / 0.178

Rating ~ Treatment * Domain

	rating
Predictors	Estimates	CI	p
(Intercept)	0.61	0.46 – 0.75	<0.001
treatmentf	-0.04	-0.25 – 0.17	0.721
domain interrupt	0.53	0.32 – 0.74	<0.001
domain [online]	0.68	0.50 – 0.87	<0.001
domain stealing	0.35	0.14 – 0.56	0.001
domain [texting]	0.79	0.60 – 0.97	<0.001
treatmentf × domain interrupt	0.13	-0.19 – 0.46	0.421
treatmentf × domain [online]	0.74	0.48 – 1.00	<0.001
treatmentf × domain stealing	-0.01	-0.33 – 0.32	0.970
treatmentf × domain [texting]	0.07	-0.19 – 0.33	0.602
Random Effects
σ²	1.07
τ₀₀ _ID2	0.31
ICC	0.22
N _ID2	496
Observations	2443
Marginal R² / Conditional R²	0.103 / 0.301

3. Analysis of appropriateness of punishment if friend vs. non-friend

Now I look at the appropriateness ratings of punishments if the violator is a friend vs. non friend.

3.1 Vizualization

Again, visual inspection does not reveal any striking differences bewteen treatments and domains.

Overall appropriatess across domains

Appropriateness by domain

Appropriateness of punishment by treatment and punisment type

3.1 ANOVA

First, I run an ANOVA to get an idea of differences in mean ratings across treatments and domains.

There was no significant main effect of treatment.

In contrast, domain was significant, suggesting that the ratings varied meaningfully across different domains.

The interaction between treatment and domain was also significant, implying that the effect of treatment differed depending on the domain.

##                    Df Sum Sq Mean Sq F value Pr(>F)    
## treatment           1      0    0.00   0.002 0.9611    
## domain              4    475  118.69  68.455 <2e-16 ***
## treatment:domain    4     15    3.83   2.208 0.0656 .  
## Residuals        7284  12630    1.73                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

3.2 Regressions

Now I fit 3 linear regressions to better explore directions of effects.

First, only with treatment as independent variable, then I add interactions with domain (insult as reference) and finally I also include punishment type as exploratory variable (angry_remark as reference).

The main results are [see tabs below for full results]:

Model 1: no effect

Model2 :

There are differences between baseline ratings across domains, but no interactions {.underline}

Model 3:

The domain “texting” was associated with significantly lower ratings compared to the reference domain.

There a few significant interactions:

The punishment type gsip has lower ratings than the reference group.
The combination of “interrupt” domain and “avoid” punishment type was associated with lower ratings compared to the reference groups.
The combination of “stealing” domain and “avoid” punishment type was associated with higher ratings compared to the reference groups.
The combination of “texting” domain and “gossip” punishment type was associated with higher ratings compared to the reference groups.
The combination of “stealing” domain and “gossip” punishment type was also associated with higher ratings compared to the reference groups.

Rating ~ Treatment

	rating
Predictors	Estimates	CI	p
(Intercept)	2.54	2.48 – 2.60	<0.001
treatmentf	-0.01	-0.06 – 0.05	0.833
Random Effects
σ²	1.56
τ₀₀ _ID2	0.24
ICC	0.13
N _ID2	496
Observations	7294
Marginal R² / Conditional R²	0.000 / 0.131

Rating ~ Treatment * Domain

	rating
Predictors	Estimates	CI	p
(Intercept)	2.61	2.50 – 2.72	<0.001
treatmentf	0.01	-0.15 – 0.16	0.936
domain interrupt	-0.20	-0.35 – -0.04	0.012
domain [online]	-0.14	-0.27 – -0.01	0.030
domain stealing	0.36	0.21 – 0.52	<0.001
domain [texting]	-0.41	-0.53 – -0.28	<0.001
treatmentf × domain interrupt	-0.01	-0.26 – 0.23	0.907
treatmentf × domain [online]	0.16	-0.02 – 0.34	0.078
treatmentf × domain stealing	-0.09	-0.33 – 0.16	0.491
treatmentf × domain [texting]	-0.09	-0.27 – 0.09	0.320
Random Effects
σ²	1.49
τ₀₀ _ID2	0.24
ICC	0.14
N _ID2	496
Observations	7294
Marginal R² / Conditional R²	0.037 / 0.171

Rating ~ Treatment * Domain* punishment type

	rating
Predictors	Estimates	CI	p
(Intercept)	2.89	2.73 – 3.05	<0.001
treatmentf	-0.11	-0.34 – 0.12	0.361
domain interrupt	-0.11	-0.34 – 0.12	0.355
domain [online]	-0.16	-0.38 – 0.05	0.131
domain stealing	0.14	-0.09 – 0.37	0.231
domain [texting]	-0.50	-0.71 – -0.28	<0.001
punishment type avoid	-0.15	-0.36 – 0.07	0.175
punishment type [gossip]	-0.69	-0.91 – -0.48	<0.001
treatmentf × domain interrupt	0.17	-0.18 – 0.52	0.329
treatmentf × domain [online]	0.27	-0.04 – 0.57	0.084
treatmentf × domain stealing	0.09	-0.26 – 0.44	0.605
treatmentf × domain [texting]	0.04	-0.27 – 0.34	0.816
treatmentf × punishment type avoid	0.17	-0.13 – 0.47	0.264
treatmentf × punishment type [gossip]	0.17	-0.13 – 0.47	0.271
domain interrupt × punishment type avoid	-0.41	-0.71 – -0.11	0.007
domain [online] × punishment type avoid	-0.08	-0.38 – 0.22	0.614
domain stealing × punishment type avoid	0.31	0.01 – 0.61	0.041
domain [texting] × punishment type avoid	-0.04	-0.34 – 0.26	0.813
domain interrupt × punishment type [gossip]	0.15	-0.15 – 0.45	0.325
domain [online] × punishment type [gossip]	0.15	-0.15 – 0.45	0.327
domain stealing × punishment type [gossip]	0.36	0.06 – 0.66	0.018
domain [texting] × punishment type [gossip]	0.30	0.00 – 0.60	0.050
(treatmentf × domain interrupt) × punishment type avoid	-0.22	-0.65 – 0.20	0.303
(treatmentf × domain [online]) × punishment type avoid	-0.21	-0.64 – 0.22	0.338
(treatmentf × domain stealing) × punishment type avoid	-0.22	-0.65 – 0.20	0.300
(treatmentf × domain [texting]) × punishment type avoid	-0.10	-0.52 – 0.33	0.661
(treatmentf × domain interrupt) × punishment type [gossip]	-0.34	-0.77 – 0.08	0.115
(treatmentf × domain [online]) × punishment type [gossip]	-0.11	-0.54 – 0.32	0.619
(treatmentf × domain stealing) × punishment type [gossip]	-0.31	-0.74 – 0.11	0.152
(treatmentf × domain [texting]) × punishment type [gossip]	-0.28	-0.71 – 0.14	0.195
Random Effects
σ²	1.44
τ₀₀ _ID2	0.24
ICC	0.15
N _ID2	496
Observations	7294
Marginal R² / Conditional R²	0.070 / 0.205

Scenarios Analyses

2025-07-08

1. Overview of scenario data

Stealing

Phone

Interrupt

Insult

Avoid

1.1. Correlation of appropriantess of violations and appropriateness of punishment.

Without separating by domain

Separating by domain

full table of correlation coefficients

2. Analysis of appropriateness of violations across all domains

2.1 ANOVA

2.2 Regressions

Rating ~ Treatment

Rating ~ Treatment * Domain

3. Analysis of appropriateness of punishment if friend vs. non-friend

3.1 Vizualization

Overall appropriatess across domains

Appropriateness by domain

Appropriateness of punishment by treatment and punisment type

3.1 ANOVA

3.2 Regressions

Rating ~ Treatment

Rating ~ Treatment * Domain

Rating ~ Treatment * Domain* punishment type